SYSTEM and METHOAD FOR PREPROCESSING CAPSULE ENDOSCOPIC IMAGE

ABSTRACT

A system and a method for preprocessing capsule endoscope images are provided. The capsule endoscopic image preprocessing system include an in vitro image removal module, an invalid image removal module, a digestive tract image classification module, a lesion and anatomical structure identification module, and a lesion and anatomical structure redundant image removal module. The capsule endoscopic image preprocessing system removes in vitro images and invalid images from capsule endoscopic images, classifies the capsule endoscopic images according to different parts of the digestive tract, identifies lesion and anatomical structures in the classified capsule endoscopic images; and removes redundant lesion and anatomical structure images according to the lesion and anatomical structures.

CROSS-REFERENCE OF RELATED APPLICATIONS

This application claims priority to Chinese Patent Application No.201710267329.8 filed on Apr. 21, 2017, the contents of which areincorporated by reference herein.

FIELD OF INVENTION

The present invention relates to the technical field of computer aideddetection, and in particular to a system and method for preprocessingcapsule endoscopic images.

BACKGROUND

In the operating procedures of existing wireless capsule endoscope, thelooks and brief information (e.g. name, gender, cellphone number) of asubject, and the capsule information (serial number, battery level,etc.) are required to be captured in vitro. In addition, the levels ofproficiency that operators have in instrument and capsule endoscope aredifferent, and the levels of adaptation of subjects to capsule endoscopeare different, so that the capsule may capture a large number of imagesin vitro.

These images prevent examination data from being inconsistent with thesubject and facilitate data management. However, these in vitro imagesare useless for doctors in image reading. The main purpose of doctors isto examine whether there are abnormalities in the digestive tract ofsubject. A large amount of in vitro images can affect the efficiency ofdoctor in image reading.

For lesions (such as bleeding, polyps, ulcers, tumors, etc.) andspecific anatomical structures (such as cardia, pylorus, etc.), thewireless capsule endoscope can capture multiple images continuously inthe digestive tract, resulting in image redundancy. This also affectefficiency of doctor in image reading.

During image capturing, empty stomach and distance between gastric wallsmay cause the images captured too bright or too dark. As a result, moreinvalid images can be obtained, affecting the efficiency of doctor inimage reading.

SUMMARY OF THE INVENTION

An object of the present disclosure is to provide a capsule endoscopicimage preprocessing system and method.

In order to solve the technical problem, the present disclosurediscloses a capsule endoscopic image preprocessing system, comprising anin vitro image removal module, an invalid image removal module, adigestive tract image classification module, a lesion and anatomicalstructure identification module, and a lesion and anatomical structureredundant image removal module; wherein the in vitro image removalmodule removes in vitro images from capsule endoscopic images based onaverage grayscale values of the capsule endoscopic image, values of themost frequent color of the capsule endoscopic images, and area ratio ofthe most frequent color of the capsule endoscopic images; wherein theinvalid image removal module removes invalid images from the capsuleendoscopic images from which the in vitro images have been removed,wherein the invalid images are images which brightness is not in apreset brightness range; wherein the digestive tract imageclassification module classifies the capsule endoscopic images accordingto different parts of the digestive tract; wherein the lesion andanatomical structure identification module identifies lesion andanatomical structures in the classified capsule endoscopic images; andwherein the lesion and anatomical structure redundant image removalmodule removes redundant lesion and anatomical structure imagesaccording to the lesion and anatomical structures.

A method for preprocessing capsule endoscopic image using the system asdescribed above is provided, comprising following blocks:

block S1: removes in vitro images from the capsule endoscopic imagesbased on average grayscale values of the capsule endoscopic images,values of the most frequent color of the capsule endoscopic images, andarea ratio of the most frequent color of the capsule endoscopic images;wherein the in vitro images is removed by:

process the first N frames of images to obtain the average grayscalevalue sequences of RGB channels M^(R)(p₁, p₂, . . . , p_(i), . . . ,p_(N)), M^(G) (p₁, p₂, . . . , p_(i), . . . , p_(N)), and M^(B) (p₁, p₂,. . . , p_(i), . . . , p_(N)), the value sequences of the most frequentcolor C(p₁, p₂, . . . , p_(i), . . . , p_(N)), and the area ratiosequences of the most frequent color S(p₁, p₂, . . . , p_(i), . . . ,p_(N)), wherein p_(i) is image frame number;

determine whether the capsule endoscope has entered the body based onthe value sequences of the most frequent color C(p₁, p₂, . . . , p_(i),. . . , p_(N)) to obtain an initial position 1; wherein the initialposition is obtained by: determine whether the value of the mostfrequent color C(p_(c)) in the image frame number of p_(c) is less thana set threshold TC, calculate the number of images MC with the value ofthe most frequent color C(p_(c+j)) continuously less than TC after theimage frame number p_(c) when the value of the most frequent colorC(p_(c)) is less than TC, determine that the capsule has entered thebody of subject when MC is greater than a set threshold TM1, and recordthe image frame number p as an initial position 1; wherein p_(c+j)represents the image after p_(c), and the value of j is [1, M];

determine whether the capsule endoscope is outside the body based on thearea ratio sequences of the most frequent color S(p₁, p₂, . . . , p_(i),. . . , p_(N)) to obtain an initial position 2; wherein the initialposition 2 is obtained by: process from the N^(th) image forward todetermine whether the maximum color value area ratio of the capsuleendoscopic image in the frame number p_(s) is greater than a setthreshold TS, calculate the number of images MS with the area ratio ofthe most frequent color S(p_(s-q)) continuously greater than TS ahead ofthe image frame number p_(s) when the area ratio of the most frequentcolor S(p_(s)) is greater than the set threshold TS, determine that theimage is an in vitro image when MS is greater than a set threshold TM2,and record the image frame number p_(c) as the initial position 2;wherein p_(s-q) represents the image ahead of p_(s), and the value of sis [1, M_(a)], wherein M_(a)=TM2;

identify in-vitro images based on changes in average grayscale values ofthe RGB channels from the image frame number p_(s) to the image framenumber p_(c); wherein the in-vitro images are identified by: calculatethe changes in average grayscale value of the RGB channels from theimage frame number p_(s) to the image frame number p_(c) by a formula ofD^(o)(p_(m))=M^(o)(p_(m))−M^(o)(p_(m+1)), wherein, o represents channeland the o channel o comprises channel R, channel G and channel B; p_(m)is an image frame number between p_(s) and p_(c) and the value of m is[s,c−1], in which, s is the frame number p_(s) and c−1 is the framenumber p_(c); determine whether the changes in average grayscale valuesof RGB channels D^(R)(p_(m)), D^(G)(p_(m)) and D^(B) (p_(m)) comply witha set threshold TD, determine that the image frame number p_(m) nearestto the image frame number p s is the dividing position for in-vivo andin vitro images when D^(R)(p_(m))<TD, D^(G)(p_(m))<TD andD^(B)(p_(m))<TD;

block S2: remove invalid images from the capsule endoscopic images fromwhich the in vitro images have been removed, wherein the invalid imagesis removed by:

convert the capsule endoscopic RGB image from which in vitro images havebeen removed to grayscale image gray, determine that a pixel is brightpixel when pixel gray(x,y) of the grayscale image gray is greater thanYH, determine that a pixel is dark pixel when pixel gray(x,y) of thegrayscale image gray is less than YL, tally the sum of number of brightpixels and dark pixels SHL, wherein SHL=sYL+sYH, in which sYL representsthe number of dark pixels and sYH represents the number of brightpixels; and remove the grayscale image gray when SHL is greater than ST,wherein ST=0.7*SI, in which SI is the total number of pixels in thegrayscale image;

block S3: use deep learning method based on a convolutional neuralnetwork model to classify the capsule endoscopic images according todifferent parts of the digestive tract;

block S4: use deep learning method to identify lesion and anatomicalstructure in the capsule endoscopic images;

block S5: identify lesion and anatomical structure images sequences fromthe capsule endoscopic image according to the lesion and anatomicalstructure, and retain images from the lesion and anatomical structureimage sequences based on the position, size and contrast characteristicsof lesion and anatomical structure, wherein the images are retained by:

calculate the score RP of the position of lesion and anatomicalstructure in each lesion and anatomical structure image of the lesionand anatomical structure images sequence, wherein the score RPrepresents the distance from lesion and anatomical structure to thecenter of corresponding image. and wherein the score RP is calculatedby:

${{RP}_{i} = {1 - \frac{\sqrt{\left( {{lx}_{i} - \frac{W}{2}} \right)^{2} + \left( {{ly}_{i} - \frac{H_{1}}{2}} \right)^{2}}}{\frac{\sqrt{W_{2} + H_{1}^{2}}}{2}}}},$

wherein, W and H₁ represent the width and height of the lesion andanatomical structure image, i represents the serial number in the lesionand anatomical structure images sequence, RP_(i) represents the score ofthe position of lesion and anatomical structure in the i^(th) image, andlx_(i) and ly_(i) represent the center coordinates of the position oflesion and the anatomical structure in the i^(th) lesion and theanatomical structure image;

calculate the score RS of the size of lesion and anatomical structure ineach lesion and anatomical structure image of the lesion and anatomicalstructure images sequence, wherein the score RS is calculated by:RS_(i)=SW_(i)×SH_(i), wherein, RS_(i) represents the score of the sizeof lesion and anatomical structure in the i^(th) lesion and anatomicalstructure image of the lesion and anatomical structure images sequence,and SW_(i) and SH_(i) represent the width and height of lesion andanatomical structure in the i^(th) lesion and anatomical structureimage;

calculate the score RC of the region contrast of lesion and anatomicalstructure in each lesion and anatomical structure image of the lesionand anatomical structure images sequence, and wherein the score RC iscalculated by:

${{RC}_{i} = {\sum\limits_{\delta}{{\delta \left( {j,k} \right)}^{2}{P_{\delta}\left( {j,k} \right)}}}},$

wherein, RC_(i) represents the score of region contrast of lesion andanatomical structure in the i^(th) lesion and anatomical structure imageof the lesion and anatomical structure images sequence,δ(j,k)=|gray(j)−gray(k)| represents the grayscale difference betweenneighboring pixels j and k, and P_(δ)(j,k) represent the probability ofoccurrence of the grayscale difference δ(j,k);

calculate the total score RT of lesion and anatomical structure image,wherein the score RT is calculated by:

RT _(i) =RP _(i) ×RS _(i) ×RC _(i),

wherein RT_(i) represents the total score RT of the i^(th) image; and

select the image with the maximum RT value in the lesion and anatomicalstructure images sequence as the image to be retained after removal ofredundancy.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a block diagram of one embodiment of a capsule endoscopicimage preprocessing system.

FIG. 2 shows a flowchart of one embodiment of a method foridentification of in vitro images.

FIG. 3 shows a deep learning model for digestive tract imagesclassification.

FIG. 4 shows a deep learning model for digestive tract images targetidentification.

FIG. 5 shows a flowchart of one embodiment of a method for removal oflesion and anatomical structure redundant images.

DETAILED DESCRIPTION

The present disclosure, including the accompanying drawings, isillustrated by way of examples and not by way of limitation. It shouldbe noted that references to “an” or “one” embodiment in this disclosureare not necessarily to the same embodiment, and such references mean atleast one.

In general, the word “module,” as used herein, refers to logic embodiedin hardware or firmware, or to a collection of software instructions,written in a programming language. In one embodiment, the programlanguage may be Java, C, or assembly. One or more software instructionsin the modules may be embedded in firmware, such as in an EPROM. Themodules described herein may be implemented as either software and/orhardware modules and may be stored in any type of non-transitorycomputer-readable medium or other storage device. Some non-limitingexamples of non-transitory computer-readable media include CDs, DVDs,flash memory, and hard disk drives.

An image preprocessing system and its preprocessing method are describedin detail below. Elements in the drawings are

-   -   1 In vitro image removal module    -   2 Invalid image removal module    -   3 Digestive tract image classification module    -   4 Lesion and anatomical structure identification module    -   5 Lesion and anatomical structure redundant image removal module

FIG. 1 shows a block diagram of one embodiment of a capsule endoscopicimage preprocessing system. In one embodiment, the capsule endoscopicimage preprocessing system comprises an in vitro image removal module 1,an invalid image removal module 2, a digestive tract imageclassification module 3, a lesion and anatomical structureidentification module 4, and a lesion and anatomical structure redundantimage removal module 5. The data output end of the in vitro imageremoval module 1 is connected to the data input end of the invalid imageremoval module 2. The data output end of the invalid image removalmodule 2 is connected to the data input end of the digestive tract imageclassification module 3. The data output end of the digestive tractclassification module 3 is connected to the data input end of the lesionand anatomical structure identification module 4. The data output end ofthe lesion and anatomical structure identification module 4 is connectedto the data input end of the lesion and anatomical structure redundantimage removal module 5.

The in vitro image removal module 1 is configured to remove in vitroimages from capsule endoscopic images based on average grayscale valuesof the capsule endoscopic images, values of the most frequent color ofthe capsule endoscopic images, and area ratio of the most frequent colorof the capsule endoscopic images, and output the removed capsuleendoscopic images to the invalid image removal module 2. The invalidimage removal module 2 is configured to further process the capsuleendoscopic images from which the in vitro images have been removed toremove invalid images, wherein the invalid images are images whichbrightness is not in the preset brightness range, and outputs theprocessed capsule endoscopic images to the digestive tract imageclassification module 3. The digestive tract image classification module3 is configured to classify the processed capsule endoscopic imagesaccording to different parts of the digestive tract, and output theclassified capsule endoscopic images to the lesion and anatomicalstructure identification module 4. The lesion and anatomical structureidentification module 4 is configured to detect the classified capsuleendoscopic images, and identify lesion and anatomical structures in theclassified capsule endoscopic images, and output the identified lesionand anatomical structures to the lesion and anatomical structureredundant image removal module 5. The lesion and anatomical structureredundant image removal module 5 is configured to remove the redundantlesion and anatomical structure images according to the lesion andanatomical structures.

The average grayscale value of a capsule endoscopic image is calculatedby:

$M^{o} = {\frac{1}{W \times H_{1}}{\sum\limits_{x = 1}^{W}{\sum\limits_{y = 1}^{H}{I^{o}\left( {x,y} \right)}}}}$

Wherein, W and H₁ represent the width and height of the capsuleendoscopic image; I^(o)(x,y) represents the grayscale value of ochannels at the position image coordinates x and y. o channels representR, G and B3 channels, and M^(o) represents the mean value of o channels.

In one embodiment, a value of the most frequent color of the capsuleendoscopic image is obtained as follows:

RGB channels in the capsule endoscopic image are converted into HSVchannels. The conversion formula is:

$H = \left\{ \begin{matrix}{0,} & {{{if}\mspace{14mu} \max} = \min} \\{{60 \times \frac{G - B}{\max - \min}},} & {{{if}\mspace{14mu} \max} = {{r\mspace{14mu} {and}\mspace{14mu} g} \geq b}} \\{{{60 \times \frac{G - B}{\max - \min}} + 360},} & {{{if}\mspace{14mu} \max} = {{r\mspace{14mu} {and}\mspace{14mu} g} < b}} \\{{{60 \times \frac{B - R}{\max - \min}} + 120},} & {{{if}\mspace{14mu} \max} = g} \\{{{60 \times \frac{R - G}{\max - \min}} + 240},} & {{{if}\mspace{14mu} \max} = b}\end{matrix} \right.$

Wherein, max represents the maximum value among RGB channels; minrepresents the minimum value among RGB channels; the formula is used tocalculate the value of hue H among HSV channels.

A histogram of the hue (H) among HSV channels is made statistics. Thestatistical formula is:

hist_(k)(H(x,y))=hist_(k-1)(H(x,y))+1

Wherein, hist represents image histogram, H(x,y) represents the value ofhue H at the position (x,y), k represents iterations (k<NUM, NUMrepresents the number of image pixels of the capsule endoscopic image).

The histogram hist(H(x,y)) of hue (H) among HSV channels is performedmedian filtering to remove interference.

A position corresponding to the maximum value Gaussian coefficient ofthe filtered histogram hist(H(x,y)) is obtained. The value of the colorcorresponding to the position is the value C of the most frequent colorof the capsule endoscopic image.

In one embodiment, the maximum value Gaussian coefficient of thefiltered histogram hist(H(x,y)) is calculated by: Perform non-linearleast-square fitting for the histogram hist(H(x,y)) with Gaussian model.The fitting formula is:

${f({hist})} = {\sum\limits_{k = 1}^{X}{a_{k}e^{- {(\frac{{hist} - c_{k}}{b_{k}})}^{2}}}}$

Wherein, a_(k) represents coefficient of the k^(th) Gaussian model, andthe value of k is 1˜X; b_(k) represents variance of the k^(th) Gaussianmodel; c_(k) represents mean value of the k^(th) Gaussian model; Xrepresents the number of Gaussian models; take a_(k), and c_(k)corresponding to the maximum coefficient a_(k) (k is 1˜X) is the value Cof the most frequent color of the capsule endoscopic image.

In one embodiment, area ratio of the most frequent color of the capsuleendoscopic image is obtained by: Binarize the hue H in HSV channels toobtain a binary image HB, wherein the threshold of binarization is TH(for example, TH=20), set the image to 1 as in vitro image if abinarization value of the image is greater than TH; otherwise set it to0 as in vivo image. Accordingly, the area ratio of the most frequentcolor is:

$S = {\frac{1}{W \times H_{1}}{\sum\limits_{x = 1}^{W}{\sum\limits_{y = 1}^{H}{{HB}\left( {x,y} \right)}}}}$

Wherein, W and H₁ represent the width and height of the binary image; xand y represent pixel coordinates, and HB represents the binary image.

In one embodiment, the in vitro image removal module 1 removes in vitroimages from the capsule endoscopic images based on the average grayscalevalues, the values of the most frequent color and the area ratio of themost frequent color. The in vitro image is removed as follows:

First, the in vitro image removal module 1 processes the first N framesof the capsule endoscopic images to obtain the average grayscale valuesequences of RGB channels M^(R)(p₁, p₂, . . . , p_(i), . . . , p_(N)),M^(G)(p₁, p₂, . . . , p_(i), . . . , p_(N)), and M^(B) (p₁, p₂, . . . ,p_(i), . . . , p_(N)), the value sequences of the most frequent colorC(p₁, p₂, . . . , p_(i), . . . , p_(N)), and the area ratio sequences ofthe most frequent color S(p₁, p₂, . . . , p_(i), . . . , p_(N)). p_(i)is image frame number. According to statistics on clinical case data,the value of N will not exceed 1000. In the embodiment, N=500, the imagecaptured by the capsule endoscope just entering the body is generallyred in hue, and H value of the red image is relatively small.Accordingly, threshold can be set to determine whether the capsuleendoscope has entered the body and identify in vitro images to exclude.

Whether the capsule endoscope has entered the body is determined basedon the value sequences of the most frequent color C(p₁, p₂, . . . ,p_(i), . . . , p_(N)) to obtain the initial position 1. The initialposition is obtained by: determine whether the value of the mostfrequent color C(p_(c)) in the image frame number of p_(c) is less thanthe set threshold TC (for example, TC=15). When the value of the mostfrequent color C(p_(c)) is less than TC, the number of images MC withthe value of the most frequent color C(p_(c+j)) continuously less thanTC after the image frame number p is calculated. When MC is greater thana preset threshold TM1 (for example, TM1=5) (in order to remove theeffect of uneven light, whether the value of the most frequent colormeet the preset threshold TM1 is determined), the capsule has enteredthe body of subject. The image frame number p is recorded as the initialposition 1; p_(c+j) represents the image after p_(c). The value of j is[1, M], wherein M=preset threshold TM1. The initial position 1 is thefirst image captured in vivo.

Whether the capsule endoscope is outside the body is determined based onthe area ratio sequences of the most frequent color S(p₁, p₂, . . . ,p_(i), . . . , p_(N)) to obtain the initial position 2. The initialposition 2 is obtained by: process from the N^(th) image forward todetermine whether the area ratio of the most frequent color in the framenumber p_(s) is greater than a set threshold TS (for example, TS=0.5)(In the body, the proportion of red part of the capsule endoscopic imageis larger; whereas outside the body, the proportion of red part of thecapsule endoscopic image is smaller.). When the area ratio of the mostfrequent color S(p_(s)) is greater than the set threshold TS, the numberof images MS with the area ratio of the most frequent color S(p_(s-q))continuously greater than TS ahead of the image frame number p_(s) iscalculated. When MS is greater than a set threshold TM2 (for example,TM2=5), the capsule endoscopic image is an in vitro image. The imageframe number p_(s) is recorded as the initial position 2; p_(s-q)represents the image ahead of p_(s). The value of s is [1, M_(a)],wherein M_(a)=TM2. The capsule endoscopic images ahead of the initialposition 2 are in vitro images.

As shown in FIG. 2, the determination of whether capsule endoscope hasentered the body of subject based on the value sequences of the mostfrequent color and the determination of whether capsule endoscope isoutside the body of subject based on the area ratio sequences of themost frequent color are performed concurrently to reduce time andimprove efficiency.

From the image frame number p_(s) to the image frame number p_(c),in-vitro images are identified based on changes in the average grayscalevalues of the RGB channels. The in-vitro images are identified by:calculate the changes in average grayscale values of the RGB channelsfrom the image frame number p_(s) to the image frame number p_(c) by aformula of D^(o)(p_(m))=M^(o)(p_(m))−M^(o)(p_(m+1)), wherein, orepresents channel and the o channel comprises channel R, channel G andchannel B; p_(m) is an image frame number between p_(s) and p_(c) andthe value of m is [s,c−1], wherein, s is the frame number p_(s) and c−1is the frame number p_(c); determine whether the changes in averagegrayscale values of RGB channels D^(R)(p_(m)), D^(G)(p_(m)) and D^(B)(p_(m)) comply with a set threshold TD. When D^(R)(p_(m))<TD, D^(G)(p_(m))<TD and D^(B) (p_(c))<TD, the image frame number p_(m) nearest tothe image frame number p is the dividing position for in-vivo images andin vitro images. The capsule endoscopic images ahead of the frame numberp_(m) are all in vitro images and can be removed.

In one embodiment, the determination of whether capsule endoscope hasentered the body of subject based on the value sequences of the mostfrequent color C(p₁, p₂, . . . , p_(i), . . . , p_(N)) and thedetermination of whether capsule endoscope is outside the body ofsubject based on the area ratio sequences of the most frequent colorS(p₁, p₂, . . . , p_(i), . . . , p_(N)) are performed concurrently toreduce time and improve efficiency.

In one embodiment, the invalid image removal module 2 further processesthe capsule endoscopic images from which the in vitro images have beenremoved to remove invalid images, wherein the invalid images are imageswhich brightness is not in the preset brightness range. The invalidimages are removed by:

Convert the capsule endoscopic RGB images from which in vitro imageshave been removed to grayscale image gray, and judge each pixelgray(x,y) of the grayscale image gray. When gray(x,y)>YH, the currentpixel is too bright; when gray (x,y)<YL, the current pixel is too dark.YH and YL are manually set empirical parameters (for example, YH=220,YL=50). Tally the sum of number of too bright and too dark pixels SHL;SHL=sYL+sYH, wherein sYL represents the number of dark pixels and sYHrepresents the number of bright pixels; when SHL>ST, the currentgrayscale image gray is too bright or too dark and is needed to beremoved; when sYL>sYH, the grayscale image is too dark, otherwise, thegrayscale image is too bright. ST is a manually set empirical parameter,related to the grayscale image size. ST=0.7*SI, wherein SI is the totalnumber of pixels in the grayscale image.

The digestive tract image classification module 3 uses a deep learningmethod based on a Convolutional Neural Network (CNN) model (such asgoolenet) to classify the capsule endoscopic images according todifferent parts of the digestive tract. The deep learning methodextracts image features by the CNN model, as shown in FIG. 3. SoftMaxfunction is used in fully connected layers to classify the features ofthe capsule endoscopic images. As a result, the digestive tract isclassified into esophagus, stomach, small intestine, and largeintestine.

The lesion and anatomical structure identification module 4 uses a deeplearning method to identify lesion and anatomical structure in thecapsule endoscopic images. As shown in FIG. 4, the feature map iscalculated through convolutional layer, and the target positions of thesuspected lesion and anatomical structure are selected on the featuremap. Then, the features of suspected targets are extracted andclassified to obtain the lesion and anatomical structures and theirclassifications.

In one embodiment, for a lesion and anatomical structure, multipleimages can be captured continuously. In order to reduce the number ofimages to be read by doctors, redundant images need to be removed, withclearer images of lesion and anatomical structure retained. The lesionand anatomical structure redundant image removal module 5 identifieslesion and anatomical structure images sequences from the capsuleendoscopic image according to the lesion and anatomical structure, andretains images from the lesion and anatomical structure image sequencesbased on the position, size and contrast characteristics of lesion andanatomical structure. The specific steps for retaining the images are asfollows, as shown in FIG. 5:

The score RP of the position of lesion and anatomical structure iscalculated in each lesion and anatomical structure image of the lesionand anatomical structure images sequence. The score RP represents thedistance from lesion and anatomical structure to the center ofcorresponding image. The score RP is calculated as follows:

${{RP}_{i} = {1 - \frac{\sqrt{\left( {{lx}_{i} - \frac{W}{2}} \right)^{2} + \left( {{ly}_{i} - \frac{H_{1}}{2}} \right)^{2}}}{\frac{\sqrt{W^{2} + H_{1}^{2}}}{2}}}},$

wherein, W and H₁ represent the width and height of the lesion andanatomical structure image, i represents the serial number in the lesionand anatomical structure images sequence, RP_(i) represents the score ofthe position of lesion and anatomical structure in the i^(th) image, andlx_(i) and ly_(i) represent the center coordinates of the position oflesion and the anatomical structure in the i^(th) lesion and anatomicalstructure image. The closer the lesion and anatomical structure are tothe center of corresponding image, the higher score RP the image can beobtained, and it needs to retain the image; conversely, the farther thelesion and anatomical structure are to the center of correspondingimage, the lower score RP the image can be obtained, and it needs toretain the image.

The score RS of the size of lesion and anatomical structure iscalculated in each lesion and anatomical structure image of the lesionand anatomical structure images sequence. The score RS is calculated asfollows:

RS _(i) =SW _(i) ×SH _(i)

wherein, RS_(i) represents the score of the size of lesion andanatomical structure in the i^(th) lesion and anatomical structure imageof the lesion and anatomical structure images sequence, and SW_(i) andSH_(i) represent the width and height of lesion and anatomical structurein the i^(th) lesion and anatomical structure image. The score RS isnormalized to [0, 1]. The bigger size the lesion and anatomicalstructure region has, the higher score RS the image can be obtained, andit needs to retain the image; conversely, the smaller size the lesionand anatomical structure region has, the lower score RS the image can beobtained, and it needs to retain the image.

The score RC of the region contrast of lesion and anatomical structureis calculated in each lesion and anatomical structure image of thelesion and anatomical structure images sequence. The score RC iscalculated as follows:

${RC}_{i} = {\sum\limits_{\delta}{{\delta \left( {j,k} \right)}^{2}{P_{\delta}\left( {j,k} \right)}}}$

Wherein, RC_(i) represents the score of region contrast of lesion andanatomical structure in the i^(th) lesion and anatomical structure imageof the lesion and anatomical structure images sequence,δ(j,k)=|gray(j)−gray(k)| represents the grayscale difference betweenneighboring pixels j and k, and P_(δ)(j,k) represents the probability ofoccurrence of the grayscale difference δ(j,k). The score RC isnormalized to [0, 1]. The higher contrast the lesion and anatomicalstructure region has, the bigger score RC the image can be obtained, andit needs to retain the image; conversely, the lower contrast the lesionand anatomical structure region has, the smaller score RC the image canbe obtained, and it needs to retain the image.

The total score RT of each lesion and anatomical structure image iscalculated. The score RT is calculated as follows:RT_(i)=RP_(i)×RS_(i)×RC_(i), wherein RT_(i) represents the total scoreRT of the i^(th) image. The bigger size the lesion and anatomicalstructure region in the image has, the closer the region is to thecenter the image, and the higher contrast the region has, the biggerscore RT the image can be obtained.

The image with the maximum RT value is selected in the lesion andanatomical structure images sequence as the image to be retained afterremoval of redundancy. That is, only one most significant image isretained for each lesion and anatomical structure. It is helpful toretain images of the lesions and anatomical structures images close tothe center of image, and with a bigger size and higher contrast, toeffectively remove redundant images.

A method for preprocessing capsule endoscopic images using the systemdescribed above, comprises following blocks. Depending on theembodiment, additional blocks may be added, others removed, and theordering of the blocks may be changed.

Block S1: The in vitro image removal module 1 removes in vitro imagesfrom the capsule endoscopic images based on average grayscale values ofthe capsule endoscopic images, values of the most frequent color of thecapsule endoscopic images, and area ratio of the most frequent color ofthe capsule endoscopic images, and outputs the removed capsuleendoscopic images to the invalid image removal module 2.

The in vitro images is removed from the capsule endoscopic image data asfollows:

First, the in vitro image removal module 1 processes the first N framesof the capsule endoscopic images to obtain the average grayscale valuesequences of RGB channels: M^(R)(p₁, p₂, . . . , p_(i), . . . , p_(N)),M^(G)(p₁, p₂, . . . , p_(i), . . . , p_(N)), M^(B)(p₁, p₂, . . . ,p_(i), . . . , p_(N)), the value sequences of the most frequent colorC(p₁, p₂, . . . , p_(i), . . . , p_(N)), and the area ratio sequences ofthe most frequent color S(p₁, p₂, . . . , p_(i), . . . , p_(N)). p_(i)is image frame number.

The in vitro image removal module 1 determines whether the capsuleendoscope has entered the body based on the value sequences of the mostfrequent color C(p₁, p₂, . . . , p_(i), . . . , p_(N)) to obtain theinitial position 1. The initial position is obtained by: determinewhether the value of the most frequent color C(p_(c)) in the image framenumber of p_(c) is less than the set threshold TC. When the value of themost frequent color C(p_(c)) is less than TC, the number of images MCwith the value of the most frequent color C(p_(c+j)) continuously lessthan TC after the image frame number p_(c) is calculated. When MC isgreater than a preset threshold TM1 (for example, TM1=5), the capsulehas entered the body of subject. The image frame number p_(c) isrecorded as the initial position 1; p_(c+j) represents the image afterp_(c). The value of j is [1, M], wherein M=threshold TM1. The initialposition 1 is the first image captured in vivo.

The in vitro image removal module 1 determines whether the capsuleendoscope is outside the body based on the area ratio sequences of themost frequent color S(p₁, p₂, . . . , p_(i), . . . , p_(N)) to obtainthe initial position 2. The initial position 2 is obtained by: processfrom the N^(th) image forward to determine whether the area ratio of themost frequent color in the frame number P is greater than the setthreshold TS (for example, TS=0.5). When the area ratio of the mostfrequent color S(p_(s)) is greater than the set threshold TS, the numberof images MS with the area ratio of the most frequent color S(p_(s-q))continuously greater than TS ahead of the image frame number p iscalculated. When MS is greater than a set threshold TM2 (for example,TM2=5), the capsule endoscopic image is an in vitro image. The imageframe number p_(s) is recorded as the initial position 2; p_(s-q)represents the image ahead of p_(s). The value of s is [1, M_(a)],wherein M_(a)=TM2. The capsule endoscopic images ahead of the initialposition 2 are in vitro images.

From the image frame number p_(s) to the image frame number p_(c),in-vitro images are identified based on changes in the average grayscalevalues of the RGB channels. The in-vitro images are identified by:calculate the changes in average grayscale values of the RGB channelsfrom the image frame number p_(s) to the image frame number p_(c) by aformula of D^(o)(p_(m))=M^(o)(p_(m))−M^(o)(p_(m+1)), wherein, orepresents channel and the o channel comprises channel R, channel G andchannel B; p_(m) is an image frame number between p_(s) and p_(c) andthe value of m is [s,c−1], wherein, s is the frame number p_(s) and c−1is the frame number p_(c); determine whether the changes in averagegrayscale values of RGB channels D^(R)(p_(m)), D^(G)(p_(m)) and D^(B)(p_(m)) comply with the set threshold TD (for example, TD=15). WhenD^(R)(p_(m))<TD, D^(G)(p_(m))<TD and D^(B)(p_(m))<TD, the image framenumber p_(m) nearest to the image frame number p_(s) is the dividingposition for in-vivo and in vitro images. The capsule endoscopic imagesahead of the frame number p_(m) are all in vitro images and can beremoved.

Block S2: The invalid image removal module 2 further processes thecapsule endoscopic images from which the in vitro images have beenremoved to remove the invalid images, wherein the invalid images areimages which brightness is not in the preset brightness range. Theinvalid images is removed by:

Convert the capsule endoscopic RGB image from which in vitro images havebeen removed to grayscale image gray, and judge each pixel gray(x,y) ofthe grayscale image gray. When gray(x,y)>YH, the current pixel is toobright; when gray (x,y)<YL, the current pixel is too dark. YH and YL aremanually set empirical parameters (for example, YH=220, YL=50). Tallythe sum of number of too bright and too dark pixels SHL; SHL=sYL+sYH,wherein sYL represents the number of dark pixels and sYH represents thenumber of bright pixels; when SHL>ST, the current grayscale image grayis too bright or too dark and needs to be removed; when sYL>sYH, theimage is too dark, otherwise, the image is too bright. ST is a manuallyset empirical parameter, related to the grayscale image size. ST=0.7*SI,wherein SI is the total number of pixels in the grayscale image.

Block S3: The digestive tract image classification module 3 uses a deeplearning method based on a Convolutional Neural Network (CNN) model(such as goolenet) to classify the capsule endoscopic images accordingto different parts of the digestive tract.

Block S4: The lesion and anatomical structure identification module 4uses a deep learning method to identify lesion and anatomical structurein the capsule endoscopic images.

Block S5: The lesion and anatomical structure redundant image removalmodule 5 identifies lesion and anatomical structure images sequencesfrom the capsule endoscopic image according to the lesion and anatomicalstructure, and retains images from the lesion and anatomical structureimage sequences based on the position, size and contrast characteristicsof lesion and anatomical structure. The specific steps for retaining theimages are as follows:

The lesion and anatomical structure redundant image removal module 5calculates the score RP of the position of lesion and anatomicalstructure in each lesion and anatomical structure image of the lesionand anatomical structure images sequence. The score RP represents thedistance from lesion and anatomical structure to the center ofcorresponding image. The score RP is calculated as follows:

${{RP}_{i} = {1 - \frac{\sqrt{\left( {{lx}_{i} - \frac{W}{2}} \right)^{2} + \left( {{ly}_{i} - \frac{H_{1}}{2}} \right)^{2}}}{\frac{\sqrt{W^{2} + H_{1}^{2}}}{2}}}},$

wherein, W and H₁ represent the width and height of the lesion andanatomical structure image, i represents the serial number in the lesionand anatomical structure images sequence, RP_(i) represents the score ofthe position of lesion and anatomical structure in the i^(th) image, andlx_(i) and ly_(i) represent the center coordinates of the position oflesion and the anatomical structure in the i^(th) lesion and theanatomical structure image. The closer the lesion and anatomicalstructure are to the center of corresponding image, the higher score RPthe image can be obtain; conversely, the lower score RP the image can beobtain.

The lesion and anatomical structure redundant image removal module 5calculates the score RS of the size of lesion and anatomical structurein each lesion and anatomical structure image of the lesion andanatomical structure images sequence. The score RS is calculated asfollows:

RS _(i) =SW _(i) ×SH _(i)

wherein, RS_(i) represents the score of the size of lesion andanatomical structure in the i^(th) lesion and anatomical structure imageof the lesion and anatomical structure images sequence, and SW_(i) andSH_(i) represent the width and height of lesion and anatomical structurein the i^(th) lesion and anatomical structure image. The score RS isnormalized to [0, 1].

The lesion and anatomical structure redundant image removal module 5calculates the score RC of the region contrast of lesion and anatomicalstructure in each lesion and anatomical structure image of the lesionand anatomical structure images sequence. The score RC is calculated asfollows:

${RC}_{i} = {\sum\limits_{\delta}{{\delta \left( {j,k} \right)}^{2}{P_{\delta}\left( {j,k} \right)}}}$

Wherein, RC_(i) represents the score of region contrast of lesion andanatomical structure in the i^(th) lesion and anatomical structure imageof the lesion and anatomical structure images sequence,δ(j,k)=|gray(j)−gray(k)| represents the grayscale difference betweenneighboring pixels j and k, and P_(δ)(j,k) represent the probability ofoccurrence of the grayscale difference δ(j,k). The score RC isnormalized to [0, 1].

The lesion and anatomical structure redundant image removal module 5calculates the total score RT of each lesion and anatomical structureimage. The score RT is calculated as follows:

RT _(i) =RP _(i) ×RS _(i) ×RC _(i),

wherein RT_(i) represents the total score RT of the i^(th) image.

The lesion and anatomical structure redundant image removal module 5selects the image with the maximum RT value in the lesion and anatomicalstructure images sequence as the image to be retained after removal ofredundancy. That is, only one most significant image is retained foreach lesion and anatomical structure. It is helpful to retain images ofthe lesions and anatomical structures images close to the center ofimage, with a bigger size and higher contrast, to effectively removeredundant images.

All of the processes described above may be embodied in, and fullyautomated via, functional code modules executed by one or more generalpurpose processors of computing devices. The code modules may be storedin any type of non-transitory readable medium or other storage device.Some or all of the methods may alternatively be embodied in specializedhardware. Depending on the embodiment, the non-transitory readablemedium may be a hard disk drive, a compact disc, a digital video disc, atape drive or other suitable storage medium.

Although certain disclosed embodiments of the present disclosure havebeen specifically described, the present disclosure is not to beconstrued as being limited thereto.

Various changes or modifications may be made to the present disclosurewithout departing from the scope and spirit of the present disclosure.

1. A capsule endoscopic image preprocessing system, comprising an invitro image removal module, an invalid image removal module, a digestivetract image classification module, a lesion and anatomical structureidentification module, and a lesion and anatomical structure redundantimage removal module; wherein the in vitro image removal module removesin vitro images from capsule endoscopic images based on averagegrayscale values of the capsule endoscopic image, values of the mostfrequent color of the capsule endoscopic images, and area ratio of themost frequent color of the capsule endoscopic images; wherein theinvalid image removal module removes invalid images from the capsuleendoscopic images from which the in vitro images have been removed,wherein the invalid images are images which brightness is not in apreset brightness range; wherein the digestive tract imageclassification module classifies the capsule endoscopic images accordingto different parts of the digestive tract; wherein the lesion andanatomical structure identification module identifies lesion andanatomical structures in the classified capsule endoscopic images; andwherein the lesion and anatomical structure redundant image removalmodule removes redundant lesion and anatomical structure imagesaccording to the lesion and anatomical structures.
 2. The system ofclaim 1, wherein the average grayscale value of the capsule endoscopicimage is calculated by:${M^{o} = {\frac{1}{W \times H_{1}}{\sum\limits_{x = 1}^{W}{\sum\limits_{y = 1}^{H}{I^{o}\left( {x,y} \right)}}}}},$wherein, W and H₁ represent the width and height of the capsuleendoscopic image, I^(o)(x,y) represents the grayscale value of ochannels at the position image coordinates x, y, o channels comprise R,G and B channels, and M^(o) represents the mean value of o channels. 3.The system of claim 1, wherein the value of the most frequent color ofthe capsule endoscopic image is obtained as follows: convert RGBchannels in the capsule endoscopic image into HSV channels, and theconversion formula is: $H = \left\{ \begin{matrix}{0,} & {{{if}\mspace{14mu} \max} = \min} \\{{60 \times \frac{G - B}{\max - \min}},} & {{{if}\mspace{14mu} \max} = {{r\mspace{14mu} {and}\mspace{14mu} g} \geq b}} \\{{{60 \times \frac{G - B}{\max - \min}} + 360},} & {{{if}\mspace{14mu} \max} = {{r\mspace{14mu} {and}\mspace{14mu} g} < b}} \\{{{60 \times \frac{B - R}{\max - \min}} + 120},} & {{{if}\mspace{14mu} \max} = g} \\{{{60 \times \frac{R - G}{\max - \min}} + 240},} & {{{{if}\mspace{14mu} \max} = b},}\end{matrix} \right.$ wherein, max represents the maximum value amongRGB channels, min represents the minimum value among RGB channels, and Hrepresents the hue value among HSV channels; make statistics for ahistogram of the hue (H) among HSV channels, and the statistical formulais:hist_(k)(H(x,y))=hist_(k-1)(H(x,y))+1, wherein, hist represents imagehistogram, H(x,y) represents the value of hue H at the position (x,y), krepresents iterations (k<NUM, NUM represents the number of imagepixels); perform median filtering for the histogram hist(H(x,y)) of hue(H) among HSV channels to remove interference; and obtain a positioncorresponding to the maximum value Gaussian coefficient of the filteredhistogram hist(H(x,y)), wherein the value of the color corresponding tothe position is the value of the most frequent color of the capsuleendoscopic image.
 4. The system of claim 3, wherein the maximum valueGaussian coefficient of the filtered histogram hist(H(x,y)) iscalculated by: perform non-linear least-square fitting for the histogramhist(H(x,y)) with Gaussian model, and the fitting formula is:${{f({hist})} = {\sum\limits_{k = 1}^{X}{a_{k}e^{- {(\frac{{hist} - c_{k}}{b_{k}})}^{2}}}}},$Wherein, a_(k) represents coefficient of the k^(th) Gaussian model, andthe value of k is 1˜X, b_(k) represents variance of the k^(th) Gaussianmodel, X represents the number of Gaussian models; take a_(k), and c_(k)corresponding to the maximum coefficient a_(k) is the value of the mostfrequent color of the capsule endoscopic image.
 5. The system of claim4, wherein the area ratio of the most frequent color of the capsuleendoscopic image is obtained by: binarize the hue H in HSV channels toobtain a binary image HB, wherein the threshold of binarization is TH;set the binary image to 1 as in vitro image when a binarization value ofthe binary image is greater than TH; set the binary image to 0 as invivo image when a binarization value of the binary image is not greaterthan TH; wherein the area ratio of the most frequent color is calculatedby:${S = {\frac{1}{W \times H_{1}}{\sum\limits_{x = 1}^{W}{\sum\limits_{y = 1}^{H}{{HB}\left( {x,y} \right)}}}}},$wherein, W and H₁ represent the width and height of the binary image, xand y represent pixel coordinates, and HB represents the binarizedimage.
 6. The system of claim 5, wherein the in vitro images is removedfrom the capsule endoscopic images by: process the first N frames of thecapsule endoscopic images to obtain average grayscale value sequences ofRGB channels M^(R)(p₁, p₂, . . . , p_(i), . . . , p_(N)), M^(G)(p₁, p₂,. . . , p_(i), . . . , p_(N)) and M^(B)(p₁, p₂, . . . , p_(i), . . . ,p_(N)), value sequences of the most frequent color C(p₁, p₂, . . . ,p_(i), . . . , p_(N)), and area ratio sequences of the most frequentcolor S(p₁, p₂, . . . , p_(i), . . . , p_(N)), wherein p_(i) is imageframe number; determine whether the capsule endoscope has entered thebody based on the value sequences of the most frequent color C(p₁, p₂, .. . , p_(i), . . . , p_(N)) to obtain an initial position 1; wherein theinitial position is obtained by: determine whether the value of the mostfrequent color C(p_(c)) in the image frame number of p_(c) is less thana set threshold TC, calculate the number of images MC with the value ofthe most frequent color C(p_(c+j)) continuously less than TC after theimage frame number p_(c) when the value of the most frequent colorC(p_(c)) is less than TC, determines that the capsule has entered thebody of subject when MC is greater than a set threshold TM1, and recordthe image frame number p_(c) as the initial position 1, wherein p_(c+j)represents the image after p_(c), and the value of j is [1, M];determine whether the capsule endoscope is outside the body based on thearea ratio sequences of the most frequent color S(p₁, p₂, . . . , p_(i),. . . , p_(N)) to obtain an initial position 2; wherein the initialposition 2 is obtained by: process from the N^(th) image forward todetermine whether the area ratio of the most frequent color in the framenumber P is greater than a set threshold TS, calculate the number ofimages MS with the area ratio of the most frequent color S(p_(s-q))continuously greater than TS ahead of the image frame number p_(s) whenthe area ratio of the most frequent color S(p_(s)) is greater than theset threshold TS, determine that the image is an in vitro image when MSis greater than a set threshold TM2, and record the image frame numberp_(s) as the initial position 2; wherein p_(s-q) represents the imageahead of p_(s), and the value of s is [1, M_(a)], wherein M_(a)=TM2;identify in-vitro images based on changes in average grayscale values ofthe RGB channels from the image frame number p_(s) to the image framenumber p_(c), wherein the in-vitro images are identified by: calculatethe changes in average grayscale values of the RGB channels from theimage frame number p_(s) to the image frame number p_(c) by a formula ofD^(o)(p_(c))=M^(o)(p_(m))−M^(o)(p_(m+1)), wherein, o represents channeland the o channel comprise channel R, channel G and channel B, p_(m) isan image frame number between p_(s), and p_(c) and the value m of is[s,c−1], in which, s is the frame number p_(s) and c−1 is the framenumber p_(c); determine whether the changes in average grayscale valueof RGB channels D^(R)(p_(m)) D^(G)(p_(m)) and D^(B)(p_(m)) comply with aset threshold TD, and determine that the image frame number p_(m)nearest to the image frame number p_(s) is the dividing position forin-vivo and in vitro images when D^(R)(p_(m))<TD, D^(G)(p_(m))<TD andD^(B)(p_(m))<TD, and the images ahead of the frame number p_(m) are invitro images.
 7. The system of claim 6, wherein the determination ofwhether capsule endoscope has entered the body of subject based on thevalue sequences of the most frequent color C(p₁, p₂, . . . , p_(i), . .. , p_(N)) and the determination of whether capsule endoscope is outsidethe body of subject based on the area ratio sequences of the mostfrequent color S(p₁, p₂, . . . , p_(i), . . . , p_(N)) are performedconcurrently.
 8. The system of claim 6, wherein the invalid image isremoved by: convert the capsule endoscopic RGB image from which in vitroimages have been removed to grayscale image gray, determine that a pixelis bright pixel when pixel gray(x,y) of the grayscale image gray isgreater than YH, determine that a pixel is dark pixel when pixelgray(x,y) of the grayscale image gray is less than YL, tally the sum ofnumber of bright pixels and dark pixels SHL, wherein SHL=sYL+sYH, inwhich sYL represents the number of dark pixels and sYH represents thenumber of bright pixels; and remove the grayscale image gray when SHL isgreater than ST, wherein ST=0.7*SI, in which SI is the total number ofpixels in the grayscale image.
 9. The system of claim 6, wherein thelesion and anatomical structure redundant image removal moduleidentifies lesion and anatomical structure images sequences from thecapsule endoscopic image according to the lesion and anatomicalstructure, and retains images from the lesion and anatomical structureimage sequences based on the position, size and contrast characteristicsof lesion and anatomical structure, wherein the images are retained by:calculate the score RP of the position of lesion and anatomicalstructure in each lesion and anatomical structure image of the lesionand anatomical structure images sequence, wherein the score RPrepresents the distance from lesion and anatomical structure to thecenter of corresponding image. and wherein the score RP is calculatedby:${{RP}_{i} = {1 - \frac{\sqrt{\left( {{lx}_{i} - \frac{W}{2}} \right)^{2} + \left( {{ly}_{i} - \frac{H_{1}}{2}} \right)^{2}}}{\frac{\sqrt{W^{2} + H_{1}^{2}}}{2}}}},$wherein, W and H₁ represent the width and height of the lesion andanatomical structure image, i represents the serial number in the lesionand anatomical structure images sequence, RP_(i) represents the score ofthe position of lesion and anatomical structure in the i^(th) image, andlx_(i) and ly_(i) represent the center coordinates of the position oflesion and the anatomical structure in the i^(th) lesion and theanatomical structure image; calculate the score RS of the size of lesionand anatomical structure in each lesion and anatomical structure imageof the lesion and anatomical structure images sequence, wherein thescore RS is calculated by: RS_(i)=SW_(i)×SH_(i), wherein, RS_(i)represents the score of the size of lesion and anatomical structure inthe i^(th) lesion and anatomical structure image of the lesion andanatomical structure images sequence, and SW_(i) and SH_(i) representthe width and height of lesion and anatomical structure in the i^(th)lesion and anatomical structure image; calculate the score RC of theregion contrast of lesion and anatomical structure in each lesion andanatomical structure image of the lesion and anatomical structure imagessequence, and wherein the score RC is calculated by:${{RC}_{i} = {\sum\limits_{\delta}{{\delta \left( {j,k} \right)}^{2}{P_{\delta}\left( {j,k} \right)}}}},$wherein, RC_(i) represents the score of region contrast of lesion andanatomical structure in the i^(th) lesion and anatomical structure imageof the lesion and anatomical structure images sequence,δ(j,k)=|gray(j)−gray(k)| represents the grayscale difference betweenneighboring pixels j and k, and P_(δ)(j,k) represent the probability ofoccurrence of the grayscale difference δ(j,k); calculate the total scoreRT of lesion and anatomical structure image, wherein the score RT iscalculated by:RT _(i) =RP _(i) ×RS _(i) ×RC _(i) wherein RT_(i) represents the totalscore RT of the i^(th) image; and select the image with the maximum RTvalue in the lesion and anatomical structure images sequence as theimage to be retained after removal of redundancy.
 10. A method forpreprocessing capsule endoscopic image using the system of claim 1,comprising following blocks: block S1: removes in vitro images from thecapsule endoscopic images based on average grayscale values of thecapsule endoscopic images, values of the most frequent color of thecapsule endoscopic images, and area ratio of the most frequent color ofthe capsule endoscopic images; wherein the in vitro images is removedby: process the first N frames of images to obtain the average grayscalevalue sequences of RGB channels M^(R)(p₁, p₂, . . . , p_(i), . . . ,p_(N)) M^(G)(p₁, p₂, . . . , p_(i), . . . , p_(N)), and M^(B)(p₁, p₂, .. . , p_(i), . . . , p_(N)), the value sequences of the most frequentcolor C(p₁, p₂, . . . , p_(i), . . . , p_(N)), and the area ratiosequences of the most frequent color S(p₁, p₂, . . . , p_(i), . . . ,p_(N)), wherein p_(i) is image frame number; determine whether thecapsule endoscope has entered the body based on the value sequences ofthe most frequent color C(p₁, p₂, . . . , p_(i), . . . , p_(N)) toobtain an initial position 1; wherein the initial position is obtainedby: determine whether the value of the most frequent color C(p_(c)) inthe image frame number of p_(c) is less than a set threshold TC,calculate the number of images MC with the value of the most frequentcolor C(p_(c+j)) continuously less than TC after the image frame numberp_(c) when the value of the most frequent color C(p_(c)) is less thanTC, determine that the capsule has entered the body of subject when MCis greater than a set threshold TM1, and record the image frame numberp_(c) as an initial position 1; wherein p_(c+j) represents the imageafter p_(c), and the value of j is [1, M]; determine whether the capsuleendoscope is outside the body based on the area ratio sequences of themost frequent color S(p₁, p₂, . . . , p_(i), . . . , p_(N)) to obtain aninitial position 2; wherein the initial position 2 is obtained by:process from the N^(th) image forward to determine whether the maximumcolor value area ratio of the capsule endoscopic image in the framenumber p_(s) is greater than a set threshold TS, calculate the number ofimages MS with the area ratio of the most frequent color S(p_(s-q))continuously greater than TS ahead of the image frame number p_(s) whenthe area ratio of the most frequent color S(p_(s)) is greater than theset threshold TS, determine that the image is an in vitro image when MSis greater than a set threshold TM2, and record the image frame numberp_(c) as the initial position 2; wherein p_(s-q) represents the imageahead of p_(s), and the value of s is [1, M_(a)], wherein M_(a)=TM2;identify in-vitro images based on changes in average grayscale values ofthe RGB channels from the image frame number p_(s) to the image framenumber p_(c); wherein the in-vitro images are identified by: calculatethe changes in average grayscale value of the RGB channels from theimage frame number p_(s) to the image frame number p_(c) by a formula ofD^(o)(p_(m))=M_(o)(p_(m))−M^(o)(p_(m+1)), wherein, o represents channeland the o channel o comprises channel R, channel G and channel B; p_(m)is an image frame number between p_(s) and p_(c) and the value of m is[s,c−1], in which, s is the frame number p_(s) and c−1 is the framenumber p_(c); determine whether the changes in average grayscale valuesof RGB channels D^(R)(p_(m)), D^(G)(p_(m)) and D^(B)(p_(m)) comply witha set threshold TD, determine that the image frame number p_(m) nearestto the image frame number p_(s) is the dividing position for in-vivo andin vitro images when D^(R)(p_(m))<TD, D^(G)(p_(m))<TD andD^(B)(p_(m))<TD; block S2: remove invalid images from the capsuleendoscopic images from which the in vitro images have been removed,wherein the invalid images is removed by: convert the capsule endoscopicRGB image from which in vitro images have been removed to grayscaleimage gray, determine that a pixel is bright pixel when pixel gray(x,y)of the grayscale image gray is greater than YH, determine that a pixelis dark pixel when pixel gray(x,y) of the grayscale image gray is lessthan YL, tally the sum of number of bright pixels and dark pixels SHL,wherein SHL=sYL+sYH, in which sYL represents the number of dark pixelsand sYH represents the number of bright pixels; and remove the grayscaleimage gray when SHL is greater than ST, wherein ST=0.7*SI, in which SIis the total number of pixels in the grayscale image; block S3: use deeplearning method based on a convolutional neural network model toclassify the capsule endoscopic images according to different parts ofthe digestive tract; block S4: use deep learning method to identifylesion and anatomical structure in the capsule endoscopic images; blockS5: identify lesion and anatomical structure images sequences from thecapsule endoscopic image according to the lesion and anatomicalstructure, and retain images from the lesion and anatomical structureimage sequences based on the position, size and contrast characteristicsof lesion and anatomical structure, wherein the images are retained by:calculate the score RP of the position of lesion and anatomicalstructure in each lesion and anatomical structure image of the lesionand anatomical structure images sequence, wherein the score RPrepresents the distance from lesion and anatomical structure to thecenter of corresponding image. and wherein the score RP is calculatedby:${{RP}_{i} = {1 - \frac{\sqrt{\left( {{lx}_{i} - \frac{W}{2}} \right)^{2} + \left( {{ly}_{i} - \frac{H_{1}}{2}} \right)^{2}}}{\frac{\sqrt{W^{2} + H_{1}^{2}}}{2}}}},$wherein, W and H₁ represent the width and height of the lesion andanatomical structure image, i represents the serial number in the lesionand anatomical structure images sequence, RP_(i) represents the score ofthe position of lesion and anatomical structure in the i^(th) image, andlx_(i) and ly_(i) represent the center coordinates of the position oflesion and the anatomical structure in the i^(th) lesion and theanatomical structure image; calculate the score RS of the size of lesionand anatomical structure in each lesion and anatomical structure imageof the lesion and anatomical structure images sequence, wherein thescore RS is calculated by: RS_(i)=SW_(i)×SH_(i), wherein, RS representsthe score of the size of lesion and anatomical structure in the i^(th)lesion and anatomical structure image of the lesion and anatomicalstructure images sequence, and SW_(i) and SH_(i) represent the width andheight of lesion and anatomical structure in the i^(th) lesion andanatomical structure image; calculate the score RC of the regioncontrast of lesion and anatomical structure in each lesion andanatomical structure image of the lesion and anatomical structure imagessequence, and wherein the score RC is calculated by:${{RC}_{i} = {\sum\limits_{\delta}{{\delta \left( {j,k} \right)}^{2}{P_{\delta}\left( {j,k} \right)}}}},$wherein, RC_(i) represents the score of region contrast of lesion andanatomical structure in the i^(th) lesion and anatomical structure imageof the lesion and anatomical structure images sequence,δ(j,k)=|gray(j)−gray(k)| represents the grayscale difference betweenneighboring pixels j and k, and P_(δ)(j,k) represent the probability ofoccurrence of the grayscale difference δ(j,k); calculate the total scoreRT of lesion and anatomical structure image, wherein the score RT iscalculated by:RT _(i) =RP _(i) ×RS _(i) ×RC _(i) wherein RT_(i) represents the totalscore RT of the i^(th) image; and select the image with the maximum RTvalue in the lesion and anatomical structure images sequence as theimage to be retained after removal of redundancy.